Shape Tensors for Eecient and Learnable Indexing

Multi-point geometry: The geometry of 1 point in N images under perspective projection has been thoroughly investigated, identifying bilinear, trilinear, and quadrilinear relations between the projections of 1 point in 2, 3 and 4 frames respectively. The dual problem-the geometry of N points in 1 image-has been studied mostly in the context of object recognition , often assuming weak perspective or aane projection. We provide here a complete description of this problem. We employ a formalism in which multi-frame and multi-point geometries appear in symmetry: points and projections are interchangeable. We then derive bilinear equations for 6 points (dual to 4-frame geometry), trilinear equations for 7 points (dual to 3-frame geometry), and quadrilinear equations for 8 points (dual to the epipolar geometry). We show that the quadrilinear equations are dependent on the the bi-linear and trilinear equations, and we show that adding more points will not generate any new equation. Applications to reconstruction and recognition: The new equations are used to design new algorithms for the reconstruction of shape from many frames, and for learning invariant relations for indexing into a database. We describe algorithms which require matching 6 (or more) corresponding points from at least 4 images, 7 (or more) points from at least 3 images, or 8 (or more) points from at least 2 images. Unlike previous approaches, the equations developed here lead to direct and linear solutions without going through the cameras' geometry. Our nal linear shape computation uses all the available data { all points and all frames simultaneously: it uses a factorization of the matrix of invariant relations into 2 components of rank 4, a shape matrix and a coordinate-system matrix .